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adjusting one image accordingly. The images are then compared, 
preferably volume element by volume element ("voxel by voxel") , 
to detect temporal differences between historical and later 
images. A composite image is displayed with synthetic colors or 
other visual clues to emphasize the scalars in the imaged tissue, 
based on temporal differences between the historical and later 
images . — — 



Please replace the paragraph at page 5, line 16 through page 
6, line 2 with the following: 

Next, in step 52, pathological feature data is preferably 
loaded by the image processor 24 (preferably from a data network 
but alternatively by retrieval from storage or through another 
input channel) . The pathological feature data is useful for 
narrowing the region of interest for image processing, and could 
be provided in any of several useful forms. Typically, a 

physical region of interest (ROI) in the historical image has 
been previously identified by a radiologist or other 
■professional, and the coordinates of that region recorded in 
association with the historical image. In that case, the 
coordinates of the ROI are the pathological feature data. 
Alternatively, certain image shapes or characteristics, stored in 
a pathological image library, could be compared with the current 
image by image correlation to define the pathological region as 
described below in connection with FIGs . 6a -6c. A further 
alternative is to employ known automated methods for the 
detection of abnormalities in digital mammograms. For example, 
see U.S. Patent No. 5,491,627 to Zhang et al . (1996). The 
locations of any detected suspect lesions, as determined by such 
an automated method, could be used as the pathological feature 
data t o identify and locate the ROI in the imag e . 
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Please replace the paragraph between page 6, line 3 0 and 
page 7, line 9 with the following: 

Next, to obtain improved registration between the historical 
and new image, the image processor correlates the pathological 
feature set with the new image (step 58) , preferably by the 
method described below in connection with FIGs. 5a and 5b. The 
information gained from the correlation operations allows better 
registration of the new image with the historical imagery and 
defines an ROI in the new image. This approach to 'better' 
registration provides for improved temporal difference 
resolution, and makes the associate measures more accurate within 
the region of interest. The ROI thus defined is then re-scanned 
(step 60) , preferably as described above, with scanning 
ultrasound equipment but with a finer level of resolution. Finer 
resolution can be obtained by using slower scan speeds, higher 
frequency ultrasound, finer and more numerous receive sensors, or 
other techniques as known in the art. The fine scan, being 
preferably thus restricted to the ROI, can be executed in a 
shorter time than would be required to image the entire breast, at 
a fine level of resolution. 



Please replace the paragraph between page 7, line 21 and 
page 8, line 4 with the following: 

After the historical ar^d new images are aligned, they are 
compared in step 64 on a point -by-point basis, to obtain a 
temporal difference image whicHv emphasizes the temporal changes 
which have occurred. A tempora\ difference image provides a 
scalar or reinforcement metric of learning, whereas a simple 
difference image simply determinesVvhether or not a change has 
occurred. A temporal difference imageXis a reinforcement feedback 
signal indicating a scalar of how ^od/bad, how much change 
(vector), or how strongly (acceleration), an action is happening. 
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The scalars may represent, for example, a natural change, or the 
effectiveness of a treatpient . For example, if digitized two 
dimensional images are cdnsidered, the images are preferably 
first normalized to a common average value, then corresponding 
pixels are subtracted and tne temporal difference values stored 
in a corresponding positionX in a temporal difference image. 
Analogously, if three-dimensional imagery is available (which is 
greatly preferred) , voxels at \ corresponding locations in the 
early and later images are subtracted one from the other, and the 
result stored in a voxel of a temporal difference (volume) image. 
(A "voxel" is a unit of graphic information that defines a small 
volume element in three-dimensional space. It is the three- 
dimensional analog of a "pixel" whicjn defines an area element in 
two- dimensional space.) Thus, normalized before and after images 
can be subtractively compared, voxel - by - voxe 1 , to obtain a three- 
dimensional temporal difference image which represents the 
temporal differences between the bef ore\ and after images. 



Please replace the paragraph at page 8, lines 5-25 with the 
following : 



As a result of step 64, at least three images are available 
to the image processor 24: an early (historical) image; a current 
image; and a temporal difference image. These images are then 
combined and/or selected for display (step 66) in various ways, 
as directed by user input from input device 32. Finally, the 
resulting image or images are displayed (step 68) and preferably 
stored (step 70) and/or exported (step 72, preferably via the 
data network) . It is highly preferable that the imagery be coded 
to emphasize temporal differences between the earlier and later 
images. For example, the temporal difference image can be coded 
in a distinct color for display based on correlation output 
values, then superimposed on the early image, the later image, or 
a composite. Various other conventional methods could be 
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equivalent ly employed to visually emphasize temporal image 
differences. For example, in one variation the temporal 
difference image can be displayed in intermittent, flickering 
display superimposed on either of the historical or current 



differences between the earlier and later images, visually 
displays the images, and visually emphasizes these temporal 
differences . 



Please replace the paragraph at page 9, lines 16-32 with the 
following : 

The success of the above described procedure depends in part 
upon the accuracy with which the earlier and later images are 
registered (in steps 56 and 62 of FIG. 2a and 2b) before 
comparison. Slight movement or deformation of the breast tissue 
or the scanning apparatus is to be expected between the earlier 
and later scans. The movement may include translation, rotation 
about any axis, or slight compression or expansion (in addition 
to biological tissue changes) . To adequately register the images 
in step 56 or 62, therefore, a computationally practical and fast 
method of registration is preferred. A preferred method of 
registration takes advantage . of the specific computational 
abilities of an optical correlator (discussed in detail below in 
connection with FIG. 9) . This preferred method (suitable for use 
in step 64 of FIG. 2b) is best described with reference to an 
example of a particular coordinate system, to aid in 
visualization . — 

Please replace the paragraph at page 12, lines 25-37 with 
the following: 

As an alternative to collapsing the images by simple 
projection, as described above, a three-dimensional image can be 
processed to create a "discriminate filter." Such filters are 




Significantly, the invention easily detects temporal 
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known and facilitate registration by providing a correlation 
filter which a composite of many images which differ by rotation 
about an axis. This allows testing of multiple, rotated slices 
simultaneously, by correlation with a composite discriminate 
filter. Once a discriminate filter is determined with acceptable 
correlation (at least equal to a pre-defined level of 
correlation) an individual filter is then selected from the set 
of slices which was used to compose the composite. In some 
cases, this method can greatly accelerate the search for best 
correlation . _„ — — 



Please replace the paragraph at page 18, lines 3-31 with the 
following : — 

After the earlier and later ROI images are adjusted to best 
register them, they can be either combined by adding, voxel -by- 
voxel, or compared by subtracting, voxel -by -voxel . Temporal 
differences between the images are easily highlighted by changing 
the color at voxels which show high variation between the dual 
images. FIG. 8 shows an example of one typical display of a 
combined image (in this case, a composite of earlier and later 
images, with . temporal differences highlighted). The outline of 
the breast 22 is shown, with. highlighted suspected lesion 200 
(simplified for illustration) as revealed by a recent 
ultrasonogram. Other more static regions of density 2 02 and 2 03 
are shown, which would suitably be displayed in a neutral color 
or grey scale. The earlier lesion region 206 is surrounded by 
the later, larger lesion 200. Suitably the later region 200 
would be highlighted, by example by display in a conspicuous 
color such as pink. The smaller lesion 206 (visible in the 
earlier historical image) would preferably be coded in a 
different color (for example, green) . This three dimensional 
image, easily digitized and archived on computer readable media, 
provides easily readable documentation of the historical image 
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changes, and can be retrieved for evidence of tissue changes. 
The image processor can also extract slices such as that cut by 
imaginary plane 302, and display the slice as a two-dimensional 
section for detailed inspection. Preferably, the processor is 
^^O^^"^"^ programmed to respond to user input so that any slice can be 
selected for display, or the user can sweep through multiple 
slices to view details of interest. 



Please replace the paragraph between page 21, line 24 and 
page 22, line 1 with the following: 



When the input the filter images have been written to the 
input and filter ports 210 and 212, the optical correlator 
produces an output image which is a two dimensional output 
correlation pattern having an optical peak or peaks (bright spot) 
at the position of greatest correlation between the collapsed 
sonographic image and the radiographic image. The degree of 
correlation is indicated by the intensity of the output signal. 
The position of the output peak on the two-dimensional matrix of 
the correlator output CCD indicates the translations or shifts of 
the images relative to one another. The output image is read from 
the output photodetector (CCD) 214 by the image processor 24 in 
the conventional manner, typically by shifting the CCD voltage 
values out sequentially in rows (or columns) and then digitizing, 
the output levels. The peak amplitudes of the 2D array provide 
the extent of correlation (degree of correlation) between the 
images. This scalar information is a measure of the temporal 
difference, and can be used as a measure of knowledge between 
images and supports actions that lead to a successful treatment 
or evaluation. 



V 
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Please replace the paragraph at page 29, lines 1-14 with the 
following : 

An image processing system and method visually documents and 
displays changes between historical and later mammographic 
images, preferably in three dimensions. A composite image is 
created which visually emphasizes temporal differences between 
the historical and later images. Preferably three-dimensional, 
digitized images, displayable in various projections, are stored 
for archival purposes on computer readable media. An image 
processor preferably exploits an optical correlator to register 
the historical and later images accurately and provide 
correlation values as temporal scalars of the differences. The 
registered images are then compared, voxel -by -voxel , to detect 
temporal differences. The composite image is displayed with 
synthetic colors or other visual clues to emphasize apparent 
changes (for example, tumor growth or shrinkage) . 



In the Claims 

Please cancel claim 20. 

Please amend claims 1, 9, 10, 11, 16 and 19 as follows; 




1. (amended) A method of . visually documenting historical 
changes in biological tissue, comprising the steps of: 
S- (a) obtaining a first image bf a region of tissue; 

(b) after obtaining said firs\ image, obtaining a second 
image of said region of tissue; 

(c) digitally storing said fi]^st and second images as 
digitized first and second images; 

(d) spatially adjusting at leastXone of said first and 
second digitized images to spatially regi^er said images so that 
corresponding features in both images are mapped to corresponding 
positions; and \^ 

(e) creating from said first and second digitized images a 
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^ ^ derived image which visually emphasizes temporal differences 
[^^^^^^ — ^ between said first and Nsecond images, thereby visually 
^ emphasizing historical chang^ between said imag es. 



9. (amended) The mettiod of claim 1, wherein said step of 
creating a composite image c^Dmprises: 



>^ :? . Vciiiit;iiu.tiu.; xxie tne^tiod of cl 

r^/^ comparing an image \intensity at a location in said 

first image with a respectivk intensity at a corresponding 
location in said second image, a^d 

determining a temporal Xdifference image value based 
upon the temporal difference between said image intensity at said 
location in said first image and the Vespective intensity at said 
corresponding location in said seconds image . 



10. (amended) The method of claim A wherein said composite 
image visually emphasizes temporal Wage differences by 
representing various regions of said composite image in synthetic 
colors, based upon temporal image differences between the first 
and second images . 



re^t 11 



11, (amended) A method of creating a displayable composite 
mammographic image from a plurality\pf raw mammographic images, 
corresponding to earlier and lat^r ^pa^mographic images, 
comprising the steps of: A ^ 

(a) obtaining the earlier image of a reigibn of tissue; 

(b) obtaining the later image of Xsubstantially the same 
region of tissue; ^^-^.^^-V'" 

(c) deriving a temporal difference image which 
represents changes between said earlier and\ater images; and 



\ 
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^ (d) combining at leatet one of said earlier and lat 

(^Oy^^^'~^ images with said temporalVgjL&f erence image, to produce 



16. (amended) The met^hod of claim 11, wherein said step of 

. \ /. 

creating a composite image\ com^/rises : 



spatially adjusti^ag^at least one of said earlier and 
later images to aid in registering said images. 



tissue 



9. (amended) A syVtem for enhancing imagery of bodily 
s by relating earlier and later images, comprising: 



an image processosr, programmed to: (a) receive said 
earlier and later images, (eO register the earlier and later 
images by controlling an optical correlator to find a position of 
correlation between said earlierv and later images, (c) derive a 
composite image from the earlier\and later images, (d) compute 
temporal differences between said earlier and later images, and 
(e) emphasize said temporal differenc^es in said composite image; 
and 

an optical correlator coupled^to said image 
processor and arranged to correlate saidVearlier and later 
images, and to output to said image processor a cross 
correlation image which is indicative of tn^ position of 
correlation of the processed images , 



Add new claim 23 as follows: 



23. The method of claim 4, whereirr^.^4_d step of creating a 
composite image comprises : 



comparing an image intensity \at\a location in said 
first image with a respective intensity ^ a corresponding 
location in said second image, and 




